Method for autonomously weeding crops in an agricultural field

ABSTRACT

A method for weeding crops includes, at an autonomous machine: recording an image at the front of the autonomous machine; detecting a target plant and calculating an opening location for the target plant longitudinally offset and laterally aligned with the location of the first target plant; driving a weeding module to laterally align with the opening location; tracking the opening location relative to a longitudinal reference position of the weeding module; when the weeding module longitudinally aligns with the first opening location, actuating the blades of the first weeding module to an open position; recording an image proximal to the weeding module; and in response to detecting the blades of the weeding module in the open position: calculating an offset between the opening location and a reference position of the weeding module, based on the image; and updating successive opening locations calculated by the autonomous machine based on the offset.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Application claims the benefit of U.S. Provisional Application No.62/626,602, filed on 5 Feb. 2018, which is incorporated in its entiretyby this reference.

TECHNICAL FIELD

This invention relates generally to the field of agricultural implementsand more specifically to a new and useful method for autonomouslyweeding crops in an agricultural field in the field of agriculturalimplements.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a flowchart representation of a first method;

FIG. 2 is a flowchart representation of a second method;

FIG. 3 is a schematic representation of a system; and

FIG. 4 is a flowchart representation of one variation of the secondmethod.

DESCRIPTION OF THE EMBODIMENTS

The following description of embodiments of the invention is notintended to limit the invention to these embodiments but rather toenable a person skilled in the art to make and use this invention.Variations, configurations, implementations, example implementations,and examples described herein are optional and are not exclusive to thevariations, configurations, implementations, example implementations,and examples they describe. The invention described herein can includeany and all permutations of these variations, configurations,implementations, example implementations, and examples.

1. First Method

As shown in FIG. 1, a first method S100 for autonomously weeding cropsin an agricultural field includes, at an autonomous machine autonomouslynavigating along crop rows in an agricultural field: recording a firstimage of a ground area below a light module arranged proximal a front ofthe autonomous machine in Block Silo; detecting a location of a firsttarget plant based on the first image in Block S120; calculating a firstopening location for the first target plant longitudinally offset fromthe location of the first target plant and laterally aligned with thelocation of the first target plant in Block S122; driving a firstweeding module to laterally align a lateral reference position of thefirst weeding module with the first opening location, the first weedingmodule arranged in a tool housing behind the light module in Block S130;tracking the first opening location relative to a longitudinal referenceposition of the first weeding module in Block S140; in response to thelongitudinal reference position of the first weeding modulelongitudinally aligning with the first opening location, actuating theblades of the first weeding module to an open position in Block S150;recording a second image of a ground area below the tool housing inBlock S160. The first method S100 also includes, in response todetecting the blades of the first weeding module in the open position inthe second image in Block S162: calculating a first longitudinal offsetbetween the first opening location and the longitudinal referenceposition of the first weeding module, based on the second image in BlockS170; and calculating a first lateral offset between the first openinglocation and the lateral reference position of the first weeding module,based on the second image in Block S172.

The first method S100 can further include: recording a third image ofthe ground area below the light module in Block Silo; detecting alocation of a second target plant based on the third image in BlockS120; calculating a second opening location for the second target plantlongitudinally offset from the location of the second target plant andlaterally aligned with the location of the second target plant in BlockS122; calculating a corrected second opening location by offsetting thesecond opening location by the first longitudinal offset and the firstlateral offset in Block S180; driving the first weeding module tolaterally align the lateral reference position of the first weedingmodule with the corrected second opening location in Block S130;tracking the corrected second opening location relative to thelongitudinal reference position of the first weeding module in BlockS140; and, in response to the longitudinal reference position of thefirst weeding module longitudinally aligning with the corrected secondopening location, actuating the blades of the first weeding module tothe open position in Block S150; and recording a fourth image of theground area below the tool housing in Block S160. The first method canfurther include, in response to detecting the blades of the firstweeding module in the open position in the fourth image in Block S162:calculating a second longitudinal offset between the second openinglocation and the longitudinal reference position of the first weedingmodule, based on the fourth image in Block S170; and calculating asecond lateral offset between the second opening location and thelateral reference position of the first weeding module, based on thefourth image in Block S172. The first method S100 can further includeaveraging the first longitudinal offset and the second longitudinaloffset to calculate a longitudinal correction in Block S190; andaveraging the first lateral offset and the second lateral offset tocalculate a lateral correction in Block S192.

2. Second Method

As shown in FIG. 2, a second method S200 for autonomously weeding cropsin an agricultural field includes, at an autonomous machine:autonomously navigating along crop rows in an agricultural field inBlock S210; at a first time, recording a first image of a ground area ina light module arranged proximal a front of the autonomous machine inBlock S220; detecting a first target plant in the first image anddetermining a first position of a stalk of the first target plant in thefirst image in Block S230; calculating a longitudinal distance from thefirst position of the first target plant to tips of closed blades in aweeding module arranged in a tool housing behind the light module,estimating a first duration of time at which the first target plant willreach the longitudinal position of the closed blades of the firstweeding module based on the longitudinal distance and a speed of theautonomous machine at the first time, and initiating a timer for a sumof the first duration of time and an open time correction in Block S240;calculating a first lateral offset from the first position of the firsttarget plant at the first time to a lateral center of the first weedingmodule and driving the first weeding module to a lateral position offsetfrom the lateral center by a sum of the first lateral offset and lateraloffset correction in Block S250; in response to expiration of the timerat a second time, triggering blades in the first weeding module to openfor an open duration in Block S260; and, in response to conclusion ofthe open duration at a third time, triggering blades in the firstweeding module to close in Block S262.

As shown in FIG. 4, the second method S200 can also include, in BlockS270: at approximately the second time, recording a second image of theground area in the tool housing; detecting the first target plant in thesecond image; determining a second position of the stalk of the firsttarget plant in the second image; calculating a longitudinal distancefrom tips of the blades to the stalk of the first target plant in thesecond image; calculating a longitudinal difference between thelongitudinal distance and a target offset open distance; calculating arevised open time correction based on a sum of the open time correctionand a product of the longitudinal difference and a speed of theautonomous machine at the second time.

As shown in FIG. 4, the second method S200 can also include, in BlockS280: calculating a lateral distance from tips of the blades to thestalk of the first target plant in the second image; and calculating arevised lateral offset correction based on a combination of the lateraloffset correction and the lateral distance.

As shown in FIG. 4, the second method S200 can further include, in BlockS290: at approximately the third time, recording a third image of theground area in the tool housing; detecting the first target plant in thethird image; determining a third position of the stalk of the firsttarget plant in the third image; calculating a longitudinal distancefrom tips of the blades to the stalk of the first target plant in thethird image; calculating a longitudinal difference between thelongitudinal distance and a target offset close distance; calculating arevised close time correction based on a sum of the close timecorrection and a product of the longitudinal difference and a speed ofthe autonomous machine at the third time.

3. Applications

Generally, the first method S100 and the second method S200 can beexecuted by an autonomous farm implement (hereinafter an “autonomousmachine”) to automatically: navigate along rows of crops in anagricultural field; detect plants as the autonomous machine 100navigates over these plants; characterize these plants as either targetplants (i.e., desired crops) or non-target plants (e.g., weeds);localize the target plants relative to the autonomous machine 100;manipulate actuators within the autonomous machine 100 to selectivelydisturb non-target plants while leaving target plants substantiallyundisturbed; monitor accuracy of the actuators during this weedingoperation; and to update the localization process based on the monitoredaccuracy over time.

In particular, the autonomous machine 100 can draw a weeding module130—in a closed position—along crop rows to disturb and remove weedsfrom topsoil in the field during a weeding operation. To identify targetplants, the autonomous machine 100 can also: record an image (e.g., acolor photographic image, a multispectral image, a merged image frommultiple front cameras 112, etc.) of a ground area at the front of theautonomous machine 100 as the autonomous machine 100 navigates alongcrop rows in the agricultural field; detect and distinguish a targetplant in the image; determine a location of the target plant relative tothe autonomous machine 100; track the location of the target plantrelative to the autonomous machine 100; laterally align a weeding module130 of the autonomous machine 100 to the target plant; and, in responsethe location of the target plant longitudinally aligning with theweeding module 130, trigger the weeding module 130 to open to avoid thetarget plant as the weeding module 130 passes the target plant and toclose once the weeding module 130 passes the target plant. Theautonomous machine 100 can repeat this process as the autonomous machine100 passes over subsequent target plants in the agricultural field.

When tracking the location of a target plant, the autonomous machine 100can: monitor actions of the autonomous machine 100 (e.g., changes in theposition and or heading of the autonomous machine 100) relative to thetarget plants and update (e.g., 30 times a second) a coordinate systemdefining the location of located target plants relative to theautonomous machine 100. Thus, the autonomous machine 100 cancontinuously actuate its weeding modules 130 to laterally align with thelocation of the target plant before performing a weeding operationaround the target plant (e.g., removing weeds from around the targetplant).

The autonomous machine 100 can therefore detect and distinguish targetplants from weeds in images recorded by a forward camera arranged nearthe front of the autonomous machine 100, such as arranged in a lightmodule 110 that consistently illuminates a ground area ahead of a set ofweeding modules 130 in the autonomous machine 100. The autonomousmachine 100 can also: extract characteristic information from thesehigh-resolution images—such as plant quality, signs of pest pressures,geospatial location, etc.; and calculate positional (or timing)parameters for actions by the set of weeding modules 130 to remove weedsproximal to the target plants well before these target plants reachthese weeding modules 130. By thus offsetting detection andcharacterization of target plants ahead of interactions with thesetarget plants, the autonomous machine 100 can repeatably and accuratelyprepare the weeding modules 130 to remove weeds from around these targetplants and to clear the target plants (e.g., by laterally centeringthese weeding modules 130 to upcoming target plants). As the autonomousmachine 100 navigates over the agricultural field thereby closing thedistance between the target plants and the set of weeding modules 130,the autonomous machine 100 can calculate appropriate opening and closinglocations for each target plant; and trigger these weeding modules 130to open and close at the calculated positions in order to dislodgesubstantially all weeds around these target plants while leaving thesetarget plants undisturbed. The autonomous machine 100 can also: monitorthese interactions between the weeding modules 130 and the targetplants; characterize deviations between actual and target results ofthese interactions; and feed these deviations forward to adjustpositional parameters calculated for subsequent target plantsapproaching these weeding modules 130 in order to maintain a highaccuracy in these weeding module 130-target plant interactions.

In one implementation, the autonomous machine 100 can utilize a rearmounted camera attached to each weeding module 130 of the autonomousmachine 100 to characterize deviations between the intended position ofthe weeding module 130 (e.g., the position that aligns with an openingor closing location corresponding to a target plant) and the actualposition of the weeding module 130 when performing the weeding operationaround a target plant. The weeding module 130 can include fiducials(e.g., blade fiducials 134) located above the soil such that thefiducials are observable to the rear camera 136. The fiducials are alsomechanically constrained relative to the blades 132 of the weedingmodule 130 such that the fiducials move with the blades 132 andtherefore indicate whether the blades of the weeding module 130 areopen, closed, or in the process of opening or closing. The autonomousmachine 100 can then select images recorded by the rear camera 136 basedon the position of the fiducials to characterize the degree to which theweeding module 130 was aligned with: the opening location correspondingto a target plant, the closing location corresponding to a target plant,or the location of the target plant itself. The autonomous machine 100can then calculate, based on these selected images, a longitudinaloffset and a lateral offset for each location associated with the targetplant (e.g., the opening location, closing location, or actual locationof the target plant).

The autonomous machine 100 can average or otherwise summarize thelongitudinal and lateral offsets calculated over a recent window of timein order to approximate a longitudinal correction and a lateralcorrection to apply to the locations of target plants. Thus, theautonomous machine 100 can correct systematic errors in its locationdetecting, location tracking, and weeding module 130 actuationprocesses.

The autonomous machine 100 is described below as including weedingmodules 130 and executing the methods S100 and S200 to de-weed anagricultural field. However, the autonomous machine 100 can implementsimilar methods and techniques to prepare and then trigger tool modulesof other types—such as seeding, watering, fertilizing, harvesting, andpesticide modules—to apply water or fertilizer to target plants, applypesticides around these target plants, and/or to deposit seeds inparticular locations, etc.

4. Autonomous Machine

As shown in FIG. 3, the autonomous machine 100 is configured toautonomously navigate through an agricultural field while detectingtarget plants in the agricultural field and performing weeding (orother) operations on or around the identified target plants. Theautonomous machine 100 can thus define a wheeled or tracked vehicle andcan include a controller 102, a chassis 104, and a drive unit 106configured to propel the autonomous machine 100 forward. The controller102 is configured to execute either of the methods S100 and S200. Theautonomous machine 100 can also include: geospatial position sensors 108(e.g., GPS) configured to output the autonomous machine's location inspace; inertial measurement units configured to output valuesrepresenting the autonomous machine's trajectory; and/or outwardlyfacing color and/or depth sensors (e.g., color cameras, LIDAR sensors,and/or structured light cameras, etc.) configured to output images fromwhich the autonomous machine 100 can detect nearby obstacles, localizeitself within a scene, and/or contextualize a nearby scene; etc. Theautonomous machine 100 can also include an onboard navigation systemconfigured to collect data from the foregoing sensors, to elect nextactions, and to adjust positions of various actuators within theautonomous machine 100 to execute these next actions.

The autonomous machine 100 also includes a forward positioned lightmodule 110 and rearward positioned tool housing 120 containing a set oftool modules (e.g., weeding modules 130). The light module 110 isconfigured to house one or more front cameras 112 while blockingexternal light from reaching the ground and over exposing any imagesrecorded by the front facing cameras. The tool housing 120 contains oneor more laterally mobile tool modules that the autonomous machine 100can actuate in order to perform weeding or other operations on thetarget plants.

4.1 Light Module

The autonomous machine 100 can also include a light module 110 arrangedproximal the front of the autonomous machine 100. The light module 110can define an enclosed volume with a downward-facing opening andspanning one or more crop rows. The light module 110 can also includecontrollable lighting elements 114 configured to repeatably illuminate aground area directly under the opening of the light module 110.Furthermore, the light module 110 can include retractable or flexibleshades (i.e. flaps) that substantially block external ambient light fromilluminating the area enclosed within the light module 110.

4.2 Tool Housing

The autonomous machine 100 can further include a tool housing 120arranged behind the light module 110 and configured to house one or moretool modules, such as described below. The tool housing 120 isconfigured with a downward-facing opening and defines a volumesufficiently large to enclose the set of tool modules of the autonomousmachine 100 and to allow for lateral adjustment of the tool modules inresponse to variable lateral locations of target plants relative to theautonomous machine 100 as the autonomous machine 100 navigates over acrop row. Thus, the tool modules housed in the tool housing 120 canextend toward target plants via the downward-facing opening in order toperform various agricultural operations described below.

In one implementation, the tool housing 120 includes a tool receptacle124: configured to transiently receive one of various tool modules; andincluding a tool positioner 122 configured to shift the tool receptacle124 laterally within the tool housing 120 in order to laterally align anend effector of a tool module—loaded into the tool receptacle 124—withsuccessive plants in a row of crops over which the autonomous machine100 passes during operation. For example, the tool housing 120 of theautonomous machine 100 can include four (or six) tool receptacles 124.To autonomously weed a field of crops, each tool receptacle 124 in theautonomous machine 100 can be loaded with a weeding module 130. As theautonomous machine 100 passes over a field of crops, tool positioners122 in these tool receptacles 124 can be independently-controlled tolaterally align their weeding modules 130 to successive target plants incorresponding rows of crops as these weeding modules 130 selectivelyupset weeds while rendering target plants (i.e., crops) substantiallyundisturbed.

Later, to water these crops, the tool receptacles 124 can be loaded withwatering tools connected to a common water reservoir installed on theautonomous machine 100. As the autonomous machine 100 navigates alongrows of crops, the autonomous machine 100 can: independently controltool positioners 122 in these tool receptacles 124 to laterally aligneach watering tool to target plants in its corresponding crop row; andselectively trigger each watering tool to dispense water onto targetplants in their corresponding crop rows.

Similarly, to fertilize these crops, the tool receptacles 124 can beloaded with fertilizing tools connected to a common fertilizer reservoirinstalled on the autonomous machine 100. As the autonomous machine 100navigates along rows of crops, the autonomous machine 100 can:independently control tool positioners 122 in these tool receptacles 124to laterally align each fertilizing tool to target plants in itscorresponding crop row; and selectively trigger each fertilizing tool todispense fertilizer onto these target plants.

Tool receptacles 124 in the tool housing 120 can be similarly loadedwith: fertilizing tools; pesticide/herbicide tools; thinning or cullingtools; seeding tools; and/or harvesting tools; etc.

In another implementation, the tool housing 120 can include a lateralsupport beam to which the various tool modules are mounted. The toolmodules can then laterally traverse the lateral support beam in order toalign themselves with target plants passing below the tool housing 120.Each tool module mounted to the lateral support beam can be individuallyelectrically connected with the autonomous machine 100 in order toreceive positioning and operative instructions in addition to electricalpower from the autonomous machine 100. The autonomous machine 100 cantherefore direct each tool module to laterally traverse the support beamvia a motor included in the tool module and/or tool housing in order tolaterally align an end effector of the tool module with a target plantin the agricultural field.

However, the autonomous machine 100 can include any tool housing 120and/or tool module configuration that enables individual lateraladjustment of the tool modules within the tool housing 120.

4.3 Cameras

The autonomous machine 100 can also include various cameras or otheroptical sensors arranged inside the light module 110 and inside the toolhousing 120 and configured to record images of ground areas passingunder the light module 110 and the tool housing 120 as the autonomousmachine 100 autonomously navigates along crop rows within anagricultural field.

4.3.1 Front Camera

In one implementation, the autonomous machine 100 includes a frontcamera 112 (e.g., a high-resolution, high-speed RGB camera ormulti-spectral imager) arranged in the light module 110, defining afield of view spanning all or a portion of the opening of the lightmodule 110, and configured to record images (or “entry images”) ofground areas entering the light module 110 from the front of theautonomous machine 100 (i.e., ground areas that the autonomous machine100 is navigating over). The autonomous machine 100 can then analyzethese entry images to detect and distinguish crops (or “target plants”)from weeds, to calculate positions of stalks of target plants with arelatively high degree of accuracy and repeatability, and/or to extractqualities of these target plants (e.g., plant age, plant size, pestpresence, fertilizer burns, nutrient or water deficiency, etc.).

The autonomous machine 100 can also include multiple front cameras 112in the light module 110. For example, the autonomous machine 100 caninclude one front camera 112 arranged over each crop row spanned by thelight module 110, wherein each front camera 112 is configured to recordimages of a single crop row. In this example, the autonomous machine 100can process images from these front cameras 112 independently andcontrol individual tool modules trailing these front cameras 112accordingly. In another example, the autonomous machine 100 includes twoor more cameras arranged in the light module 110 and defining fields ofview that span multiple (e.g., two or three) crop rows. The autonomousmachine 100 can then: process images output by these camerasindependently, as described below; or stitch discrete, concurrent imagesrecorded by these cameras into a composite image of the ground areaspanned by the light module 110 and then process this composite image asdescribed below. The autonomous machine 100 can also include multiplecameras in the light module 110 with overlapping fields of view. Theautonomous machine 100 can then process concurrent images to obtaindepth information for the target plant or any aspect of the agriculturalfield via binocular machine vision algorithms.

However, the autonomous machine 100 can include any configuration of oneor more front cameras 112 such that the autonomous machine 100 canrecord images for accurate identification, localization, and analysis oftarget plants.

4.3.2 Rear Cameras

The autonomous machine 100 can also include one or more rear cameras 136arranged in the tool housing 120. For example, the autonomous machine100 can include one camera coupled to each tool receptacle 124 in thetool housing 120, wherein each camera faces downward to define a fieldof view that includes an end effector of a tool module loaded into itscorresponding tool receptacle 124 (or that includes a reference featureconnected to the end effector and extending above the soil when the endeffector is submerged in soil, as described below). In one example,wherein the autonomous machine 100 includes a weeding module 130 as thetool module, the end effector includes a set of weeding blades furtherdescribed below.

Each of these rear cameras 136 can thus record image data representinginteractions between a corresponding tool module and plants passingunder the tool housing 120 (e.g., a distance between the tips of weedingblades when the blades open to avoid a target plant and/or a distancebetween the tips of the blades and the target plant when the bladesclose after passing over the target plant). In particular, a rear camera136 can record images of interactions between an end effector—of a toolmodule loaded into the tool housing 120—and plants passing under theautonomous machine 100 (or “exit images”). Furthermore, since theautonomous machine 100 may analyze optical data recorded by the rearcamera(s) 136 to determine relative positions of the end effector ortool modules and plants passing through the tool housing 120 in order togauge effectiveness of the tool module in executing an action—ratherthan to extract plant quality and/or other more sophisticatedmetrics—the rear camera(s) 136 may be of lower resolution or otherwisecollect lower-density optical data than the front camera(s) 112, therebyreducing processing time or processing load for the autonomous machine100 to interpret images recorded by the rear camera(s) 136. However, therear camera(s) 136 can record images at a frame rate greater than thefront camera(s) 112 in order to enable the autonomous machine 100 todetect proximity of the blades 132 to a target plant with a high degreeof temporal accuracy and resolution.

Alternatively, the autonomous machine 100 can include anelectromechanical trigger configured to electrically trigger the rearcamera 136 to record an image in response to a position or change inposition of an end effector of a tool module (e.g., a position of theblades 132 of a weeding module). For example, the autonomous machine caninclude a rear camera 136 that is configured to record an image inresponse to the blades 132 of the weeding module reaching an openposition or a closed position.

In implementations wherein the autonomous machine 100 includes a weedingmodule 130, the rear camera 136 for the weeding module 130 can bemounted or otherwise affixed to the weeding module 130 such that thecenter of the field of view of the rear camera 136 substantially alignswith a reference position of the primary weeding blades furtherdescribed below.

In an alternative implementation wherein the autonomous machine 100includes other tool modules (e.g., a watering module, fertilizingmodule, seeding module), the autonomous machine 100 can include a rearcamera 136 facing downward with the end effector of the tool modulewithin the field of view of the rear camera 136. Thus, the autonomousmachine can include rear camera(s) 136 coupled to the tool module with afield of view encompassing the end effector of the tool module in ordercapture the particular agricultural function of the tool module.

4.4 Weeding Module

In one variation, a tool receptacle 124 in the tool housing 120 isloaded with a weeding module 130. Generally, the weeding module 130 isconfigured to adjust its lateral position relative to the autonomousmachine 100 in order to align with target plants as the autonomousmachine 100 navigates in the agricultural field. The weeding module 130also includes a set of blades 132 (e.g., weeding blades), which can beinserted at a predefined depth under the surface of the topsoil. Theblades 132 can then selectively disturb weeds around target plants byopening and closing around target plants in the agricultural field. Theweeding module 130 can also include a camera mounted directly on theweeding module 130 that is configured to record the position of bladefiducials 134 in order to determine an exact time and/or location of theblades when the blades open or closed. The autonomous machine 100 canthen analyze images taken by the camera in order to determine lateraland longitudinal corrections for locating target plants.

In one implementation, the autonomous machine 100 can access theposition of a weeding module 130 and, therefore, the position of theblades 132 (e.g., via calibration described below). The weeding module130 can include encoders on any moving parts of the module or theautonomous machine 100 such that the autonomous machine 100 can accessthe position of the weeding module 130 within the tool housing 120 atany given time.

4.4.1 Blades and Blade Actuator

In one implementation, the weeding module 130 can include a pair ofblades 132 and a blade actuator configured to transition the blades 132between open and closed positions. In this implementation, the blades132: can define curved, cantilevered sections extending from driveshaftssuspended from the tool receptacle 124; and submerged in topsoil, suchas configured to run 0-60 millimeters below grade while the autonomousmachine 100 traverses an agricultural field in order to dislodge weedsfrom topsoil. The blades 132 can also be geared or otherwise driventogether by the blade actuator—such as an electromagnetic rotary motoror a pneumatic linear actuator—such that the blades 132 open and closetogether. Alternatively, the weeding module 130 can include separateactuators and or motors configured to drive each of the blades 132separately.

In the closed position by default, tips of blades 132 can come intocontact or nearly into contact such that the blades 132 form acontinuous barricade across the width of the weeding module 130; theblades 132 in the closed position can thus displace topsoil and tearweeds out of the topsoil across the full lateral span of the blades 132in the closed position. In this implementation, the pair of blades 132can also be vertically offset relative to one another, thereby enablingthe tips of the blades to overlap to ensure a continuous barricadeacross the width of the weeding module 130 in the closed position.

However, when opened by the blade actuator, tips of the blades 132spread apart, thereby forming an open region between the tips of theblades 132. The blade actuator can therefore transition the blades 132to the open position in order to form a gap between the blades 132:sufficient to fully clear the stalk of a target plant passing under theweeding module 130; sufficient to minimally disrupt topsoil around thetarget plant; but sufficiently closed to dislodge other non-targetplants (e.g., weeds) immediately adjacent the target plant from thetopsoil as the autonomous machine 100 autonomously navigates past thetarget plant.

In this implementation, in the open position, the blade actuator canopen the blades 132 by a distance matched to a type and/or growth stageof crops in the field. For example, the target open distance between theblades 132 in the open position can be set manually by an operator priorto dispatching the autonomous machine 100 to weed the agriculturalfield. In this example, the operator can select the target open distancebetween the tips of the primary blade in the open position from adropdown menu, such as: 20 mm for lettuce at two weeks from seeding; 30mm for lettuce at three weeks from seeding; 40 mm for lettuce at fourweeks from seeding; and 50 mm spread for lettuce after five weeks fromseeding and until harvest. Alternatively, to avoid disrupting a smalltarget plant with shallow roots but to improve weeding accuracy for moremature plants with deeper root structures, the operator can select thetarget open distance between the tips of the primary blade in the openposition of: 50 mm for lettuce at two weeks from seeding; 40 mm forlettuce at three weeks from seeding; 30 mm for lettuce at four weeksfrom seeding; and 20 mm spread for lettuce after five weeks from seedingand until harvest. Alternatively, the autonomous machine 100 oraffiliated support infrastructure can automatically select thesedistances. The blade actuator can then implement these settings during anext weeding operation at the field, as described below. The driveshaftsof the blade actuator can be positioned at a distance greater than themaximum open distance of the blades 132 in order to allow target plantsthat can pass within the gap formed by the open blades 132 to also passbetween the driveshafts of the blade actuator without beingsubstantially disturbed.

Furthermore, the blade actuator can be configured to retain the blades132 in the closed position by default such that the blades 132 displacetopsoil and tear weeds out of the topsoil across the full lateral spanof the blades 132 as the autonomous machine 100 navigates along a croprow. However, in this example, upon nearing a target plant theautonomous machine 100 can trigger the blade actuator to open the blades132 by the target open distance to permit the target plant to passthrough the weeding module 130 substantially undisturbed; once thetarget plant passes, the autonomous machine 100 can trigger the bladeactuator to return to the closed position, as described below.

In one implementation, the autonomous machine 100 can configure theweeding module 130 to perform a thinning operation on an agriculturalfield. In this implementation, the blade actuator can retain the blades132 in the closed position when passing over target plants that aresmaller than a predetermined size as detected by the front camera 112.

The autonomous machine 100 can also define a reference position for theblades 132 of a weeding module 130 that the autonomous machine 100 canuse to align the blades 132 of the weeding module 130 with incomingtarget plants in order to properly clear the target plants whenperforming the weeding operation. In one implementation, the referenceposition is located at the intersection point of the blades 132 when theblades 132 are in the closed position. In another implementation, thereference position is located at the lateral center between the twoblades 132 when the blades are in the closed position.

4.4.2 Blade Geometry

Furthermore, the blades 132 can curve inwardly toward one another toenable the actuator to transition the blades 132 from the open positionto the closed position just as the tips of the blades 132 pass a targetplant, thereby causing the blades to come (nearly) into contact justbehind the target plant and thus dislodge other plants (e.g., weeds)just behind the target plant. Because the blades are curved inwardly,trailing sections of the blades 132 may remain offset from the targetplant—and therefore may not disturb the target plant—when the weedingmodule 130 rapidly closes the tips of the blades 132 just behind (e.g.,just millimeters behind) the stalk of the target plant while alsodrawing weeds out of the topsoil around the target plant as theautonomous machine 100 advances along a crop row.

For example, weeds may grow predominately around crops, particularly ifthese crops are selectively watered and fertilized. As the autonomousmachine 100 draws the blades 132 near a target plant, the blade actuatorcan maintain the blades 132 in the closed position in order to upsetweeds ahead of the target plant. However, as the tips of the bladeapproach the target plant, the blade actuator can open the blades 132 toclear the stalk of the target plant; as the tips of the blade pass thetarget plant, the blade actuator can again close the blades 132 tocontact weeds just behind the target plant and to draw these weeds outof the topsoil. In particular, the curvature of the blades 132 can bematched to a normal operating speed (e.g., two miles per hour) of theautonomous machine 100 and to a normal closing speed of the blades 132when actuated by the blade actuator: such that the tips of the blades132 come back (nearly) into contact with one another when actuated bythe blade actuator (e.g., just as the tips of the blades 132 pass thestalk of the target plant) in order to extricate weeds immediatelybehind the target plant from the topsoil; and such that the blades 132remain sufficiently offset from the target plant, thereby rendering thetarget plant and soil in the immediate vicinity of the target plantsubstantially undisturbed.

However, the blades 132 can also be configured with other geometriessuch as straight, outwardly curved, or any more complex geometry.

4.4.3 Blade Fiducials

In one variation, the weeding module 130 also includes a set of bladefiducials 134 (i.e. fiducials) arranged above the blades 132 andconfigured to visually indicate locations of the tips of the blades 132,which may be submerged in the topsoil and thus visually obscured fromthe rear camera(s) 136.

In one implementation, the weeding module 130 includes one secondaryblade offset above each primary blade, wherein each secondary bladedefines a tip (e.g., a color-coded or lighted tip) offset verticallyabove its corresponding primary blade below. The autonomous machine 100can thus track positions of tips of the blade fiducials 134—such asrelative to target plants passing under the weeding module 130—anddetermine positions of tips of the blades 132 relative to the targetplant accordingly.

Alternatively, the autonomous machine 100 can monitor positions of thetips of the blades 132 directly from outputs of sensors (e.g., opticalencoders, limit switches) coupled to driveshafts from which the blades132 are suspended.

In an alternative variation, the weeding module 130 can include bladefiducials 134 that do not directly indicate the position of the blades132 of the weeding module 130 (e.g., by occupying the same space withinthe field of view of the rear camera 136) and instead indicate thecurrent position (i.e. an open position, a closed position, or atransitional position). In this variation, the weeding module 130 caninclude shorter fiducials that are mechanically constrained to theposition of the blades 132.

4.4.4 Fixed Blades

In one variation, the autonomous machine 100 also includes fixed bladesextending between blades 132 in adjacent weeding modules 130 in the toolhousing 120. These fixed blades can be configured to disturb weedsbetween crops rows (e.g., in inter-row areas) not handled by the movableblades 132. In another variation, the fixed blades can be mounted toweeding modules in order to laterally align with the outside edges ofcrop rows and remove weeds in those areas of the agricultural field thatare in between the crop rows.

4.4.5 Depth Adjustment

In one variation, the autonomous machine 100 also includes a depthadjuster configured to adjust a depth of the blades 132 of the weedingmodule 130 in the topsoil. For example, the autonomous machine 100 can:estimate a distance from a reference point on the autonomous machine 100to topsoil under the autonomous machine 100, such as with a separatedepth sensor arranged in the tool housing 120; and implement closed-loopcontrols to actuate the depth adjuster to raise or lower the blades 132substantially in real-time in order to maintain a consistent depth ofthe blades 132 below the local surface of the topsoil based on thisdistance to the topsoil.

Alternatively, the chassis 104 of the autonomous machine 100 can besuspended on an adjustable-height suspension, and the autonomous machine100 can adjust the depth of blades 132 in the topsoil by adjusting theheight of the chassis 104 during operation.

In yet another alternative implementation, each weeding module 130 canrest on the surface of the topsoil on a set of wheels configured tosuspend the weight of the weeding module 130 above them via a suspensionsystem. The weight of the weeding module 130 can then press the blades132 to an adjustable depth (adjustable by adjusting the suspension)below the level of the wheels as they roll over the surface of thetopsoil. Thus, the suspension of the weeding module 130 itself regulatesthe depth of the blades 132 under the topsoil. Furthermore, theautonomous machine 100 can adjust the spring rate and/or damping of thesuspension automatically according to properties of the target plantsand/or weeds in the agricultural field. Alternatively, a user of theautonomous machine can adjust the spring rate and/or damping of thesuspension of the weeding module 130 in order to achieve a consistentdepth of the blades 132 under the topsoil of the agricultural field.

Yet alternatively, the depth of the blades 132 can be set manually andthen fixed when the weeding module 130 is loaded into a tool receptacle124 in the tool housing 120.

5. Rear Camera Calibration

In one implementation, the autonomous machine 100 can calibrate theposition of a rear camera 136 fixed to a weeding module 130 relative tothe reference position of the blades 132 (e.g., the blade referenceposition) of the same weeding module 130. This calibration processdetermines the pixel location of the reference position of the blades132 such that the autonomous machine 100 can accurately measure offsetsbetween intended locations of the blades 132 and actual locations of theblades 132 during weeding operation. Furthermore, the autonomous machine100 can also determine positions of the blade fiducials 134 within thefield of view of the rear camera 136 that correspond with the openposition of the blades 132 and the closed position of the blades 132.

In this implementation, the autonomous machine 100 can display a fieldof view of the rear camera 136 and provide a diagnostic interface (e.g.,via a built-in diagnostic monitor or via a monitor connected with theautonomous machine 100) for an operator of the autonomous machine 100 toinput the pixel location of the blade reference position. Additionally,the diagnostic interface enables the operator of the autonomous machine100 to specify a position of the blade fiducials 134 corresponding to anopen position of the blades 132 and corresponding to a closed positionof the blades 132.

In an alternative implementation, the autonomous machine 100 performsthe calibration process autonomously by: drawing the blades 132 out ofthe topsoil such that the blades 132 are directly visible by the rearcamera 136; locating (e.g., via computer vision algorithms) a referenceposition of the blades 132, such as by detecting the pixel location ofthe intersection point between the blades 132; detecting a closedposition of the blade fiducials 134 corresponding to a closed positionof the blades 132; actuating the blades 132 to an open position; anddetecting an open position of the blade fiducials 134 corresponding toan open position of the blades 132. The autonomous machine 100 canperform the calibration process multiple times successively in order toobtain an average or tolerance window for the reference position of theblades 132, the closed position of the blade fiducials 134, and the openposition of the blade fiducials 134.

In yet another alternative, the autonomous machine can correlate thelocation of the blade fiducials 134 with the position of the blades 132autonomously by: actuating the blades 132 to a closed position;recording successive images of the blade fiducials 134 as the autonomousmachine navigates across the agricultural field; generating a differenceimage of the successive images to locate the position of the bladefiducials 134 within the field of view of the rear camera 136 when theblades are in the closed position; actuating the blades 132 to an openposition; recording successive images of the blade fiducials 134 as theautonomous machine navigates across the agricultural field; andgenerating a difference image of the successive images to locate theposition of the blade fiducials 134 within the field of view of the rearcamera 136 when the blades are in the open position.

However, the autonomous machine 100 can detect or otherwise obtain(e.g., via demarcation by an operator of the autonomous machine 100) thepixel location of the blade reference position, the closed position ofthe blade fiducials 134, and the open position of the blade fiducials134 within the field of view of the rear camera 136.

6. Weeding Operation

When dispatched to an agricultural field to perform a weeding operation,the autonomous machine 100 can execute Blocks of the methods S100 orS200 to: autonomously navigate along crop rows in the agriculturalfield; to detect and track target plants; and to selectively actuate theweeding modules 130 to dislodge plants other than target plants from thetopsoil.

In particular, the methods S100 and S200 are described below as executedby the autonomous machine 100 when loaded with a weeding module 130. Forthe autonomous machine 100 that includes multiple tool receptacles 124,each loaded within a weeding module 130, the autonomous machine 100 canexecute multiple instances of the method S100 or of the method S200 foreach of the weeding modules 130 loaded in the autonomous machine 100 inorder: to detect target plants in multiple discrete crop rows; toindependently reposition these weeding modules 130 into lateralalignment with target plants in their corresponding crop rows; and toselectively trigger their blade actuators to open and close in order toupset weeds while leaving target plants in these rows substantiallyundisturbed.

6.1 Navigation

In one implementation, the autonomous machine 100: includes a set ofgeospatial position sensors 108 (e.g., GPS sensors); and tracks itsabsolute position and orientation within a geospatial coordinate systembased on outputs of these geospatial position sensors 108. Inpreparation for a weeding operation within an agricultural field, theperimeter or vertices of the agricultural field can be defined withinthe geospatial coordinate system and then loaded onto the autonomousmachine 100. The longitudinal direction and lateral offset of crop rowsin this agricultural field, start and stop locations (e.g., within thegeospatial coordinate system), a target ground speed, positions of knownobstacles in the agricultural field and other relevant data can besimilarly loaded onto the autonomous machine 100.

Once the autonomous machine 100 is dispatched to this agricultural fieldand once a weeding cycle by the autonomous machine 100 is subsequentlyinitiated by an operator (e.g., locally or remotely), the autonomousmachine 100 can: navigate to the specified start location (e.g., aroundrather than through the georeferenced boundary of the agriculturalfield); orient itself into alignment with the longitudinal direction ofa first set of crop rows at the start location; and accelerate to thetarget ground speed parallel to the first set of crop rows. Whiletraversing the set of crop rows, the autonomous machine 100 can: recordimages of target plants within the light module 110 in Block Silo;detect locations of target plants in Block S120; and interpolate croprows between sequential target plants in this first set of crop rows.The autonomous machine 100 can then implement closed-loop controls toveer left or veer right in order to maintain the first set of crop rowsapproximately centered within the width of the autonomous machine 100.The autonomous machine 100 can additionally or alternatively detect croprows through images recorded by outwardly-facing cameras on the front ofthe autonomous machine 100 and align itself to these crop rowsaccordingly.

Upon reaching the georeferenced boundary of the agricultural field, theautonomous machine 100 can autonomously execute a reverse-offsetmaneuver to turn 180° and align itself with a second set of croprows—adjacent and offset from the first set of crop rows by an effectivewidth of the tool housing 120 (e.g., by four crop rows for the toolhousing 120 loaded with four weeding modules 130). For example, theautonomous machine 100 can execute a U-turn maneuver responsive to bothGPS triggers and optical features indicative of the end of the crop rowin images recorded by various cameras in the autonomous machine 100. Theautonomous machine 100 can again: accelerate to the target ground speedparallel to the second set of crop rows; maintain the second set of croprows centered within the width of the autonomous machine 100; and repeatthe reverse-offset maneuver to align itself with a third set of croprows upon reaching the opposing georeferenced boundary of theagricultural field. The autonomous machine 100 can repeat theseprocesses until the autonomous machine 100 has traversed the entirety ofthe specified area of the agricultural field, then autonomously navigateback to a stop location, and finally enter a standby mode.

However, the autonomous machine 100 can implement any other method ortechnique to track its location and orientation, to autonomouslynavigate across an agricultural field, and to maintain itself inalignment with rows of crops during a weeding operation.

6.2 Plant Detection and Identification

While navigating along the row of crops, the autonomous machine 100 canregularly record entry images through one or more forward cameras (e.g.,RGB color or multispectral cameras) arranged in the light module 110,such as at a rate of 24 Hz, in Block Silo of the first method and BlockS220 of the second method. Upon receipt of a first entry image recordedby the forward camera(s) at a first time, the autonomous machine 100 canimplement computer vision techniques to: detect and extract features inthe first entry image; and to identify these features are representingtarget plants, weeds, soil, or other non-target matter in Block S120 ofthe first method and Block S230 of the second method. For example, theautonomous machine 100 can implement template matching, objectrecognition, or other plant classifier or computer vision techniques todetect plant matter in the first entry image and to distinguish a firsttarget plant from weeds in the first entry image (e.g., based on plantcolor(s), leaf shape, and/or size, etc.). The autonomous machine 100 canadditionally or alternatively implement deep learning techniques (e.g.,convolutional neural networks) to identify target plants in an entryimage.

Once the autonomous machine 100 identifies a target plant in the firstentry image, the autonomous machine 100 can also approximate orotherwise locate a stalk (or meristem) of the target plant. For example,the autonomous machine 100 can: calculate a centroid of foliage of thetarget plant and associate this centroid as the location of the stalk ofthe first target plant. Once the autonomous machine 100 determines thelocation of the stalk of the first target plant—such as in the firstentry image or relative to a reference point on the autonomous machine100—the autonomous machine 100 can: store this location of the stalk ofthe first target plant as a “first position” of the first target plantat a first time; and associate the first target plant with a first toolreceptacle 124 in the tool housing 120 in response to the first positionof the first target plant falling within a range of lateral positionscorresponding to the first tool receptacle 124. For example, for thetool housing 120 that includes four tool receptacles 124, each loadedwith a weeding module 130, the autonomous machine 100 can associate thefirst target plant with a first, rightmost weeding module 130 in thetool housing 120 if the first position of the first target plant fallswithin a right 25% of the light module 110.

In an alternative implementation, the autonomous machine 100 canidentify and associate each plant row over which the autonomous machine100 is navigating with a particular weeding module 130. The autonomousmachine 100 can then associate plants within each identified plant rowwith the weeding module 130 that has been assigned to that plant row.The autonomous machine 100 can reevaluate the relative position of eachplant row periodically (e.g., after every five target plants in theplant row).

However, the autonomous machine 100 can implement any other method ortechnique to detect and distinguish a target plant from other organicmatter represented in the first entry image and to determine a positionof the stalk of the plant in the first entry image.

6.3 Weeding Operation Alternatives

As described above, the autonomous machine 100 can maintain the blades132 in the closed position by default in order to upset all non-targetplants in the path of the blades 132 as the autonomous machine 100navigates along a row of crops while opening the blades 132 to avoiddisturbing the target plants. However, the autonomous machine 100 canexecute two methods to accomplish the above task. The autonomous machine100 can execute the first method S100 by tracking the spatial locationof the target plants and operating the blades of the weeding module 130according to the spatial alignment of the target plants with the blades132 of the weeding module 130. Alternatively, the autonomous machine 100can execute the second method S200 to estimate a time at which thetarget plant will align with the blades 132 of the weeding module 130and then operate the blades at that time. Each of these methods aredescribed further below.

The initial Blocks of the first method S100 and the second method S200are substantially similar as described above. For example, both methodsinvolve navigating along crop rows in an agricultural field; recordingentry images of target plants in the agricultural field in Blocks Siloand S220 respectively; and detecting a target plant in the entry imagesof the agricultural field in Blocks S120 and S220. However, each of themethods involves the execution of different Blocks in order toaccurately align the blades 132 of the weeding module 130 with thedetected target plants.

6.4 Spatial Weeding Operation

The autonomous machine 100 can execute the first method S100 to:calculate an opening location corresponding to a detected target plantin Block S130; track the location of a target plant in amachine-relative spatial coordinate system in Block S140; actuate theblades of the weeding module 130 in response to alignment between thelocation of the target plant and the blades of the weeding module 130 inBlock S150; and calculate lateral and longitudinal offsets for theweeding operation in Blocks S160, S162, S170 and S172. Thus, theautonomous machine 100 can continually track the spatial accuracy of theweeding operation to remove weeds from areas surrounding the targetplant without disturbing the target plant.

6.4.1 Target Plant Location Detection

Once the autonomous machine 100 identifies a target plant in the firstentry image in Block S120, the autonomous machine 100 can: identify apixel location approximating the position of the stalk of the targetplant; extract the pixel location of the first target plant in firstentry image; map the extracted pixel location to a pixel projection ofthe target plant; calculate a lateral and longitudinal location of thetarget plant relative to the autonomous machine 100 based on the depthof topsoil surface below the front camera 112.

The autonomous machine 100 identifies a pixel location approximating alocation of a stalk of the target plant within the field of view of thecamera at the time the image was recorded. For example, the autonomousmachine 100 can calculate a centroid of the set of pixels identified asthe target plant. Alternatively, the autonomous machine 100 can performadditional computer vision techniques to approximate the pixel locationof the stem from the set of pixels identified as the target plant. Theautonomous machine 100 can then extract the coordinates of theidentified pixel.

The autonomous machine 100 can then calculate a pixel projection (e.g.,a ray incident to the lens of the front camera), corresponding to theextracted pixel coordinates. The calculated pixel projection thereforecorresponds to an estimated heading, in three-dimensional space, of thestalk of the target plant relative to the front camera 112. Theautonomous machine 100 can store a mapping of each pixel location in thefield of view of the camera to a pixel projection corresponding to thatpixel. Alternatively, the autonomous machine 100 can include aparametric model, which takes in pixel coordinates and outputs the pixelprojection corresponding to the pixel coordinates based on the opticalproperties of the front camera 112.

The autonomous machine 100 can store the pixel projection correspondingto the pixel representing a target plant using any suitable mathematicaldescription (e.g., projective coordinates, vector descriptions, linearfunctions, etc.). The autonomous machine 100 can also store an origin ofa pixel projection in three-dimensional space.

In order to estimate the location of the target plant along the pixelprojection corresponding to the pixel representing the target plant, theautonomous machine 100 can estimate the depth of the plant bed at thelocation of the target plant. In one implementation, the autonomousmachine 100 references a depth sensor located within the light module110 to estimate the depth of the plant bed relative to the front camera112 of the autonomous machine 100. Additionally or alternatively, inimplementations with at least two front cameras 112, the autonomousmachine 100 can estimate the depth of the plant bed using binocularvision techniques (e.g., by comparing the pixel location of plant in twooverlapping images). In yet another implementation, the autonomousmachine 100 can estimate the depth of the plant bed based on theextension of the weeding modules 130 toward the plant bed.

Once the location coordinates of the target plant have been estimated,the autonomous machine 100 can laterally align an end effector of a toolmodule to perform an agricultural function, such as weeding at orimmediately around the target plant, such as in Block S130. Theautonomous machine 100 can also calculate opening and closing locationsbased on the location coordinates of the target plants in Block S122further described below.

However, the autonomous machine 100 can implement any other method forestimating a lateral and/or longitudinal location of the target plantand can represent any relevant location in any convenient coordinatesystem (e.g., global, relative to the autonomous machine 100, relativeto the boundaries of the agricultural field).

6.4.2 Opening and Closing Location Calculation

After the autonomous machine 100 has calculated a location of the targetplant (i.e. a location of a stalk of the target plant), the autonomousmachine 100 can calculate an opening location for the target plant and aclosing location for the target plant in Block S122. The openinglocation is a location in the coordinate system defined relative to theautonomous machine 100 that is a short distance in front of the targetplant such that when the autonomous machine 100 actuates the blades ofthe weeding module 130 into an open position at the opening location,the blades of the weeding module 130 substantially avoid disturbing thetarget plant while still removing weeds proximal to the target plant.More specifically, the opening location is longitudinally offset fromthe location of the target plant and laterally aligned with the locationof the target plant. The closing location is a location in thecoordinate system defined relative to the autonomous machine 100 that isa short distance behind the target plant such that when the autonomousmachine 100 actuates the blades of the weeding module 130 into a closedposition at the closing location the blades of the weeding module 130close just behind the target plant without disturbing the target plant.

In one implementation, the autonomous machine 100 calculates the openinglocation and the closing location based on an opening distance and aclosing distance respectively. The opening distance defines thelongitudinal offset between the location of the target plant and theopening location, and the closing distance defines the longitudinaloffset between the target plant and the closing location. In oneimplementation, the autonomous machine 100 provides an interface for anoperator to adjust the opening and/or closing distance. Alternatively,the autonomous machine 100 can: provide an interface for an operator tospecify particular properties of the plants in the agricultural fieldand/or properties of the crop row, such as plant age, plant size, soildensity; and calculate an opening and/or closing location based on theparameters specified by the operator. For example, the autonomousmachine 100 can access a lookup table for opening distances and/orclosing distances corresponding to each combination of plant age, plantsize, and soil density. In yet another implementation, the autonomousmachine 100 can: collect weeding operation data over a period of time;calculate opening distances and/or closing distances for a variety ofobservable parameters; and automatically calculate opening distancesand/or closing distances based on the characteristics of the targetplant as detected in an entry image.

In one implementation, the autonomous machine 100 can define the openingdistance and closing distance to be equal such that the location of thetarget plant is centered between the opening location for the targetplant and the closing location of the target plant.

6.4.3 Location Tracking

The autonomous machine 100 tracks the location of the target plant, theopening location corresponding to a target plant, and/or the closinglocation corresponding to a target plant in two-dimensional space (orthree-dimensional space) relative to the autonomous machine 100 based onchanges in the global position and orientation of the autonomous machine100, such that the autonomous machine 100 can: laterally align theblades 132 of the weeding module 130 to the target plant at the time theweeding module 130 reaches the target plant as in Block S130; thelongitudinal location of the target plant can be tracked as in BlockS140; and longitudinal alignment between the blades of the weedingmodule 130 and the opening location corresponding to the target plantcan be detected as in Block S150.

The autonomous machine 100 can recalculate the location of each targetplant or other relevant location relative to the autonomous machine 100periodically (e.g., 30 times a second) based on the last or mostaccurately calculated location for the target plant (e.g., from amongsta number of entry images). Each instance in which the autonomous machine100 recalculates the relative locations is hereinafter referred to as a“frame.” After calculating the various tracked locations related totarget plants, the autonomous vehicle tracks its change in globalposition and/or its change in global orientation since the most recentframe. The autonomous machine 100 can then apply spatial transformations(e.g., rotations and/or translations) to the coordinates defining thetracked locations and generate a new set of locations based on theresult of the spatial transformations. In this manner, the autonomousmachine 100 repeatedly updates the tracked locations relative to theautonomous machine 100.

In one implementation, the autonomous machine 100 can simultaneouslytrack the location of multiple target plants in several crop rows (e.g.four or six), utilizing the method described above to estimate thelocation of each target plant. In one implementation, the autonomousmachine 100 stores the estimated location of each target plant in amatrix. The autonomous machine 100 can represent the location of atarget plant in any suitable coordinate system (e.g. cartesian,spherical, cylindrical). The autonomous machine 100 can utilize imagetracking techniques to track each target plant between successive imagesand update the location estimate corresponding to each target plantaccordingly.

In one implementation, the autonomous machine 100 can record successiveimages of the plant bed while the target plant passes underneath thelight module 110, thereby continuously updating the pixel projectioncorresponding to the pixel representing the target plant. Therefore, theautonomous machine 100 can continue updating the location of the targetplant until the tool modules reach the target plant. Concurrent witheach update to the target plant's location, the autonomous machine 100can actuate the corresponding tool module to laterally align with eachupdated location as further described below.

Therefore, the autonomous machine 100 can update the location of eachtarget plant represented in the location matrix based on the mostrecently recorded image of each target plant. Thus, even when a targetplant is out of view of the front camera 112, the autonomous machine 100can use the last location of the target plant to track the target plant.

The autonomous machine 100 can also track or access the relativelocation of a reference position corresponding to each weeding module130 (e.g., to the tips of the blades 132 of the weeding module 130). Theautonomous machine 100 can access linear, rotational, or any other formof encoder to detect the location of a weeding module 130 relative tothe autonomous machine 100. The autonomous machine 100 can also accessthe location of each rear camera 136 relative to the autonomous machine100 in order to calculate locations and/or distances associated withtarget plants based on images from the rear camera(s) 136.

6.4.4 Weeding Module Positioning

After the autonomous machine 100 calculates a location of a target plantbased on an entry image, the autonomous machine 100 tracks the locationof the target plant relative to the autonomous machine 100, as describedabove. Thus, the autonomous machine 100 can drive a first weeding module130 to laterally align the reference position of the first weedingmodule 130 with the first opening location, the first weeding module 130arranged in a tool housing 120 behind the light module 110 in BlockS130. More specifically, the autonomous machine 100 can execute closedloop controls to match a lateral coordinate of the reference position ofthe weeding module 130 (corresponding to the position of the blades 132of the weeding module 130) to the most recent tracked lateral locationof the target plant or opening location by laterally actuating theweeding module 130 within the tool housing 120. In this manner, theautonomous machine 100 can continuously actuate the weeding module 130within the tool module from when the autonomous machine 100 initiallydetects a target plant until a time at which an opening location for thetarget plant is longitudinally aligned with the weeding module 130.

When tracking multiple successive target plants within the same croprow, the autonomous machine 100 can laterally align with the targetplant immediately in front of the weeding module 130 until the weedingmodule 130 has performed a weeding operation around the target plant.Once the autonomous machine 100 closes the blades of the weeding module130 behind the target plant, the autonomous machine 100 can actuate theweeding module 130 to laterally align with the next plant in the croprow.

6.4.5 Primary Blade Operation

Upon laterally aligning a reference position of the weeding module 130with an opening location corresponding to a target plant, the autonomousmachine 100 tracks the opening location relative to a longitudinalcoordinate of the reference position (e.g. a longitudinal referenceposition) of the first weeding module 130 in Block S140; and in responseto the longitudinal reference position of the first weeding module 130longitudinally aligning with the first opening location, actuating theblades of the first weeding module 130 to an open position in BlockS150. More specifically, the autonomous machine 100 opens the blades ofthe weeding module 130 that is laterally aligned with the target plantwhen the blades of the weeding module 130 are in a previously-calculatedopening location for the target plant. Thus, the autonomous machine 100can successfully avoid disturbing the target plant by opening the bladesof the weeding module 130 at the correct lateral and longitudinallocation while still removing weeds around and closely proximal to thetarget plant.

6.4.6 Interaction Image Acquisition

During the weeding operation, the autonomous machine 100 records aseries of images capturing interactions between the blades of theweeding module 130 and the target plant in order to correct errors inthe location detection, tracking, and actuation of the autonomousmachine 100. More specifically, the autonomous machine 100 executes avisual-spatial feedback system by: recording one or more images of theground below the tool housing 120 via a rear camera 136 in Block S160;and in response to detecting the blades of the first weeding module 130in the open position in the second image in Block S162: calculatingoffsets between the reference position of the blades of the weedingmodule 130 and the opening location and/or closing locationcorresponding to a target plant. Thus, the autonomous machine 100visually detects (e.g., via the rear camera 136) the actual offsetbetween the reference position of the blades of the weeding module 130and opening location corresponding to target plants at a time when theoffset between the reference location and the opening locations shouldbe equal to zero according to the detection, tracking, and actuationsteps described above.

Therefore, in order to calculate the actual offset between the referenceposition of the blades of the weeding module 130 and an opening positioncorresponding to a target plant, the autonomous machine 100 can detectthe first image, from amongst a sequence of exit images recorded by therear camera 136, in which the blades have reached an open position whileapproximately aligned with an opening location corresponding to a targetplant. More specifically, the autonomous machine 100 can: record asequence of exit images of the ground area below the tool housing 120,the sequence of images comprising an interaction image; sequentiallyanalyze the sequence of images; and, in response to detecting the bladesof the weeding module 130 in the open position in the interaction imageas the first instance of the blades being open in the sequence ofimages, select the interaction image from the sequence of images. Thus,the autonomous machine 100 selects a first image, in the sequence ofimages recorded by the rear camera 136, in which the blades of theweeding module 130 have reached the open position with which tocalculate the actual offsets between the blades of the weeding module130 and an opening position corresponding to a target plant.

In one implementation, the autonomous machine 100 can visually detectthat the blades of the weeding module 130 are in the opening position bydetecting a position of blade fiducials 134 of the first weeding module130 in the second image; and, in response to the position of the bladefiducials 134 of the first weeding module 130 in the second imageindicating the open position, detect blades of the first weeding module130 in the open position. The autonomous machine 100 can perform thisanalysis by comparing the position of the blade fiducials 134 in eachimage record by the rear camera 136 with the open position of the bladefiducials 134 determined during the calibration process described above.If, in an image, the blade fiducials 134 of the weeding module 130 arewithin a threshold range of the open position of the blade fiducials 134(e.g., the blade fiducials 134 occupy a substantially similar area ofthe field of view of the rear camera 136 to the calibrated position ofthe blade fiducials 134 that corresponds to an open position of theblade fiducials 134), then the autonomous machine 100 can detect thatthe blades of the weeding module 130 are in the open position and selectthe image as the interaction image.

In alternative implementations, the autonomous machine 100 can selectthe interaction image in other ways. For example, the autonomous machine100 can include an electrical-mechanical trigger that activates when theblades of the weeding module 130 reach the open position. Therefore, theautonomous machine 100 can select an image recorded by the rear camera136 of the weeding module 130 at substantially the same time that theelectrical mechanical trigger was activated as the interaction image.

The autonomous machine 100 can also record and analyze interactionimages corresponding to closing the blades of the weeding module 130when the reference position of the weeding module 130 is approximatelyaligned with a closing location corresponding to a target plant. Theautonomous machine 100 can thus implement the process described abovewith reference to an opening location corresponding to a target plantand an open position of the blades of the weeding module 130 andinstead: select an interaction image in a sequence of exit imagesrecorded by the rear camera 136 wherein the blades of the weeding module130 are in a closed position after the autonomous machine 100 haslongitudinally aligned with a closing location corresponding to a targetplant.

In implementations wherein the autonomous machine 100 is loaded withother tool modules, the autonomous machine 100 can select interactionimages in a similar manner. For example, the autonomous machine 100 can:detect the position of a fiducial mechanically linked to an end effectorof a tool module; and select an interaction image from a sequence ofexit images based on the detected position of the fiducial.

6.4.7 Offset Detection

Upon selecting an interaction image depicting the blades of the weedingmodule 130 opening (or closing) proximal a target plant, the autonomousmachine 100 can detect longitudinal and lateral offsets between areference position of the blades of the weeding module 130 and theopening or closing location as depicted in the interaction image inBlocks S170 and S172. For example, if the interaction image depicts theblades of a weeding module 130 opening at the opening location (althoughthe blades may not be visible in the image since they are located undera layer of topsoil), the autonomous machine 100 can calculate alongitudinal and lateral offset between the reference position of theblades based on the interaction image and the opening location based onthe interaction image. In another example, if the interaction imagedepicts the blades of a weeding module 130 closing at the closinglocation, then the autonomous machine 100 can calculate a longitudinaland lateral offset between the reference position of the blades based onthe interaction image and the closing location based on the interactionimage. Thus, the autonomous machine 100 can: detect a target plant inthe interaction image; calculate a location of the opening location ofthe target plant based on the interaction image; and calculate an offsetbetween the opening location and the reference position of the blades inthe interaction image.

More specifically, the autonomous machine 100 can: detect a pixellocation of a target plant in the interaction image; calculate a pixellocation of the opening location corresponding to the target plant basedon the pixel location of the target plant in the interaction image;calculate a longitudinal offset between the opening location and thelongitudinal reference position of the weeding module 130 based on thepixel location of the opening location and the pixel location of thelongitudinal reference position (e.g., which may be set in thecalibration process for the rear camera 136) in the interaction image;calculate a lateral offset between the opening location and the lateralreference position of the weeding module 130 based on the pixel locationof the opening location and the pixel location of the lateral referenceposition (e.g., which may be determined in the calibration process forthe rear camera 136) in the interaction image. The autonomous machine100 can perform the abovementioned calculations by executing computervision techniques based on optical properties of the rear camera 136,the orientation of the rear camera 136, and the depth of the plant bedbelow the rear camera 136.

In one implementation, the autonomous machine 100 identifies a pixellocation of a target plant (which corresponds with an estimated actuallocation of the target plant) in the interaction image by matchingfeatures in the interaction image with an entry image of the same targetplant. Thus, the autonomous machine 100 can: generate a first set ofvisual features of an entry image; detect a target plant in the entryimage; track the target plant; actuate a weeding module 130 to laterallyalign with the target plant; upon identifying an interaction image forthe target plant, generating a second set of visual features for theinteraction image; map the first set of visual features to the secondset of visual features; and detect an estimated pixel location of thefirst target plant based on the set of like features. The autonomousmachine 100 can then calculate an estimated opening location and/or aclosing location based on the image. By detecting the target plant inthe interaction image via feature mapping with an entry image of thesame plant, the autonomous machine 100 can detect the target plant morequickly (i.e. with less computational power) thereby improving detectiontime for a target plant in an interaction image and allowing for quickercalculation of longitudinal and lateral offsets.

Upon calculating the longitudinal offset and lateral offset, theautonomous machine 100 can store the offset values, and any metadataassociated with the target plant for which the offset values werecalculated (e.g., the pixel location within the field of view of thefront camera 112 with which the location of the target plant wascalculated, the lateral location of the target plant relative to thecentral axis of the autonomous vehicle, the ambient temperature proximalthe autonomous machine 100, etc.). The autonomous machine 100 can theninput the stored offset values into a correction model further describedbelow.

6.4.8 Correction Model

The autonomous machine 100 executes a correction model based oncalculated offset values within a recency window or buffer in order tocorrect for errors in calibration, location detection, tracking, andactuation of the weeding module 130 in Blocks S190 and S192. As theautonomous machine 100 navigates the agricultural field, the autonomousmachine 100 identifies target plants, and performs weeding operationsaround target plants thereby detecting, calculating, and storinglongitudinal offsets and lateral offsets for each detected target plant,as described with reference to Blocks Silo, S120, S122, S130, S140,S150, S160, S162, S170, and S172. The autonomous machine 100 can theninput a subset of the stored offset values (e.g., a buffer of the last10, 100, 1000, etc. offset values) into a correction model to calculatea longitudinal correction and a lateral correction for successivetracked locations in the agricultural field.

In one implementation, the autonomous machine 100 executes a correctionmodel that outputs a running average of the offset values within therecency window. Additionally or alternatively, the autonomous machine100 can implement a correction model that calculates a separate runningaverage for various regions of the field of view of the front camera 112in which the target plant was initially detected. For example, upondetecting a target plant in a first region of the field of view of thefront camera(s) 112 and performing a weeding operation around the targetplant, the autonomous machine 100 can categorize the offset valuescalculated in relation to the target plant based on the detection regionin order to apply more accurate longitudinal corrections and lateralcorrections to locations tracked by the autonomous machine 100. In oneimplementation, the detection regions can be laterally divided sectionsof the field of view of the front camera 112.

In another implementation, the autonomous machine 100 executes a machinelearning model trained on the offset data and the metadata associatedwith each offset data point. The autonomous machine 100 can train themachine learning model to predict a longitudinal correction and alateral correction for any location calculated by the autonomous machine100 (e.g., target plant locations, opening locations, closing locations,etc.) based on metadata associated with the location. The metadata caninclude any number of parameters characterizing the target plant or thecondition of the autonomous machine 100 and/or the agricultural field,such as soil density, ambient temperature, region of the camera field ofview within which a target plant was initially detected, and the speedof the autonomous machine 100 etc. Based on metadata measured oraccessed in relation to a target plant, the correction model can outputa longitudinal correction and a lateral correction for a location of thetarget plant and/or opening or closing locations associated therewith.

However, the autonomous machine 100 can calculate a longitudinalcorrection and a lateral correction based on calculated offsets betweenthe reference position of the weeding module 130 and various locationstracked by the autonomous machine 100 in any other way.

6.4.9 Location Correction

Upon calculating a longitudinal correction and/or a lateral correction,the autonomous machine 100 can apply the corrections to locationscalculated in Blocks S120 and S122 in real-time. For example, theautonomous machine 100 can record longitudinal offsets and lateraloffsets while executing a weeding operation on a first target plant andapply corrections based on the recorded offsets when calculating thelocation of the subsequent target plant. The autonomous machine 100 cantherefore: calculate a corrected opening location by offsetting theopening location by a longitudinal correction and a lateral correctionin Block S180. In one implementation, the autonomous machine 100 cancalculate the corrected location of a target plant by offsetting acalculated location of a target plant by both the longitudinalcorrection and the lateral correction. Alternatively, the autonomousmachine 100 can apply the corrections when calculating opening andclosing locations for an uncorrected location of a target plant. Thus,when calculating an opening location for a target plant and/or a closinglocation for a target plant, the autonomous machine 100 can offset theinitially calculated opening location or closing location by thelongitudinal correction and the lateral correction.

6.5 Temporal Weeding Operation

The autonomous machine 100 can execute the second method S200 to:autonomously navigate along crop rows in an agricultural field in BlockS210; record an entry image of a ground area at the front of theautonomous machine 100 in Block S220; detect a target plant in the entryimage in Block S230; calculate a longitudinal distance from the firstposition of the first target plant to tips of closed blades 132 in aweeding module 130 arranged in a tool housing 120 behind the lightmodule 110, estimating a first duration of time at which the firsttarget plant will reach the longitudinal position of the closed blades132 of the first weeding module 130 based on the longitudinal distanceand a speed of the autonomous machine 100 at the first time, andinitiating a timer for a sum of the first duration of time and an opentime correction in Block S240; calculate a first lateral offset from thefirst position of the first target plant at the first time to a lateralcenter of the first weeding module 130 and driving the first weedingmodule 130 a lateral position offset from the lateral center by a sum ofthe first lateral offset and lateral offset correction in Block S250; inresponse to expiration of the timer at a second time, trigger blades 132in the first weeding module 130 to open for an open duration in BlockS260; and, in response to conclusion of the open duration at a thirdtime, trigger blades 132 in the first weeding module 130 to close inBlock S262. Thus, the autonomous machine 100 can temporally control theweeding operation to remove weeds from areas surrounding the targetplant without disturbing the target plant.

6.5.1 Plant Time and Lateral Position Prediction

Once the autonomous machine 100 detects a target plant and the locationof its stalk in the first entry image, the autonomous machine 100 canpredict: a time that target plant will reach a corresponding weedingmodule 130 based on a speed of the autonomous machine 100 and alongitudinal position of the target plant in the first entry image inBlock S240; and a lateral position of the weeding module 130 that willpermit the blades 132 of the weeding module 130 to clear the targetplant when in the open position based on a lateral position of targetplant in the first entry image in Block S250.

In one implementation, the autonomous machine 100 calculates its realground velocity from a most-recent sequence of GPS locations andorientations read from GPS sensors in the autonomous machine 100.Alternatively, the autonomous machine 100 can calculate the velocity ofa target plant—relative to the autonomous machine 100—based on changesin the detected position of the target plant over a sequence of entryimages recorded by the forward camera(s) as the target plant passesthrough the light module 110. Yet alternatively, the autonomous machine100 can determine its velocity relative to the ground—and thereforerelative to a target plant—based on its wheel speed and wheel position;and/or based on dead reckoning to interpret outputs of an IMU arrangedin the autonomous machine 100.

The autonomous machine 100 can then: calculate a longitudinal distancefrom the first position of the first target plant to tips of the blades132—in the closed position—at the first time, such as based on a knownposition of the first weeding module 130 in the first tool receptacle124 relative to the light module 110; and then divide this longitudinaldistance by a current speed of the autonomous machine 100 to calculate afirst estimate of a duration of time—from the current time—at which thefirst target plant will reach the longitudinal position of the closedblades 132 of the first weeding module 130 in Block S240. The autonomousmachine 100 can also calculate a first lateral offset from the firstposition of the first target plant at the first time to a lateral center(or “home position”) of the first weeding module 130 in Block S250, asshown in FIG. 2. (The autonomous machine 100 can also implement thisprocess continuously to estimate this duration of time.)

In a similar implementation, the autonomous machine 100 can: define areference tool plane coincident tips of the blades 132 in the closedposition and parallel to the front plane of the autonomous machine 100;extrapolate an arc traversed by the first target plant—relative to theautonomous machine 100—from the first position at the first time to areference plane based on the current trajectory of the autonomousmachine 100; calculate a length of the arc; and estimate a future timeat which the first target plant will reach the reference tool plane (andtherefore reach the blades 132 of the first weeding module 130) based onthe velocity of the autonomous machine 100. The system can alsocalculate a first lateral offset from: an intersection of the are andthe reference tool plane; and an intersection of the reference toolplane and a longitudinal axis passing through the lateral center of thefirst weeding module 130.

The autonomous machine 100 can then: sum the first time estimate with anopen time correction—described below—to calculate an adjusted open time;sum the first lateral offset with a lateral offset correction—describedbelow—to calculate an adjusted lateral offset; initiate an open timerfor this adjusted open time; and trigger the tool positioner 122 in thefirst tool receptacle 124 to drive the first weeding module 130laterally to the adjusted lateral offset.

Furthermore, the autonomous machine 100 can modify the adjusted opentime based on a change in velocity of the autonomous machine 100 sincethe last time estimate was calculated. More specifically, the autonomousmachine 100 can modify the adjusted open time based on a change in speedof a weeding module 130 relative to a next target plant, which may be afunction of (a change in) the velocity of the autonomous machine 100 inthe longitudinal direction, (a change in) the angular velocity of theautonomous machine 100, and a position of the weeding module 130 insidethe autonomous machine 100. Similarly, the autonomous machine 100 canmodify the adjusted lateral offset based on a change in position of theautonomous machine 100 since the last lateral offset was calculated.More specifically, the autonomous machine 100 can modify the adjustedlateral offset based on a change in position of a weeding module 130relative to a next target plant, which may be a function of the angularvelocity of the autonomous machine 100 over this period of time and aposition of the weeding module 130 inside the autonomous machine 100.

6.5.2 Plant Time and Lateral Position Revision

In one variation, the system can repeat the foregoing Blocks of thesecond method S200 and techniques to recalculate the time estimate forarrival of the first target plant at the plane of the first weedingmodule 130 in Block S240 and to recalculate the lateral offset from thefirst target plant to the centerline of the first weeding module 130 inBlock S250 when the first target plant is detected in subsequent entryimages recorded by the forward camera(s).

For example, the forward camera can regularly record color entry imagesof a ground area bounded by the light module 110 at a rate of 24 Hz.Upon receipt of a second entry image—following the first entryimage—from the forward camera at a second time, the autonomous machine100 can implement object tracking techniques to again detect the firsttarget plant—now at a second position within the light module 110—inthis second entry image. The autonomous machine 100 can then repeat theforegoing processes to: calculate a revised estimate of a future time atwhich the first target plant will reach the longitudinal position of theclosed blades 132 of the first weeding module 130; and calculate arevised lateral offset for the first weeding module 130 based on adifference between the second position of the first target plant at thesecond time and the lateral center of the first weeding module 130. Theautonomous machine 100 can then: confirm that the current and precedingtime estimates of lateral offsets are converging; apply the open timecorrection to the revised time estimate; reset the open timer to thecorrected revised time estimate; apply the lateral offset correction tothe revised lateral offset; and trigger the tool positioner 122 in thefirst tool receptacle 124 to move the first weeding module 130 to thisrevised, adjusted lateral position.

The autonomous machine 100 can continue to implement this process basedon entry images recorded by the forward camera(s) until the first targetplant exits the light module 110.

6.5.3 Target Plant Avoidance

When the open timer expires, the autonomous machine 100 can: trigger theblade actuator in the first weeding module 130 to open the blades 132 inBlock S260, thereby permitting the stalk of the first target plant andadjacent topsoil to pass through the gap between the blades 132substantially undisturbed; and set a close timer for a corrected closeduration (e.g., a sum of default close duration and a close timecorrection, described below). Once the close timer expires, theautonomous machine 100 can trigger the blade actuator in the firstweeding module 130 to close the blades 132 in Block S262, therebybringing the tips of the blades 132 back into (near) contact just aft ofthe first target plant and enabling the blades 132 to upend weedsimmediately behind the first target plant without upsetting the firsttarget plant itself, as shown in FIGS. 2 and 4.

6.5.4 Primary Blade Interaction Tracking

As the first target plant enters the tool housing 120 and approaches thefirst weeding module 130, the autonomous machine 100 triggers the bladeactuator in the first weeding module 130 to open and close the blades132 responsive to expiration of the open and close timers, respectively.Before, during, and/or after this actuation of the blades 132, a rearcamera 136 over the first weeding module 130 can record a sequences ofexit images (e.g., a “burst” of exit images). The autonomous machine 100can then implement methods and techniques similar to those describedabove to detect a first target plant (e.g., a stalk of the first targetplant) in these exit images and to estimate a location of the stalk ofthe first target plant accordingly.

The autonomous machine 100 can also detect and track positions of theblade fiducials 134 or other physical fiducials coupled to blades 132 inthis sequence of exit images and track the positions of the blades 132accordingly, as described above.

The autonomous machine 100 can then: extract a lateral distance from acenterline between the blades 132 and the center of the stalk of thefirst target plant from an exit image recorded by the rear camera 136 ator just before the autonomous machine 100 triggered the blade actuatorto open the blades 132 in the first weeding module 130 (e.g., at or justbefore the open timer expires at a second time); subtract the firstlateral offset—currently occupied by the first weeding module 130—fromthe lateral distance to calculate a new lateral offset correction; andthen store this new lateral offset correction for implementation by thefirst weeding module 130 for a next target plant passing through thefirst weeding module 130 in Block S280. Alternatively, the autonomousmachine 100 can compare these exit images to identify a first exit imagethat depicts initial opening of the blades 132 and implement similarprocesses to a new lateral offset correction from this first exit image.(Once the first target plant passes the first weeding module 130 andbefore the next target plant reaches the first weeding module 130, theautonomous machine 100 can adjust the lateral position of the firstweeding module 130 according to the new lateral offset correction.)

The autonomous machine 100 can similarly: calculate longitudinaldistance from tips of the blades 132 to the center of the stalk of thefirst target plant as the blades 132 approach the first target plant andat or just before the autonomous machine 100 triggers the blade actuatorto open the blades 132 (e.g., at or just before the open timer expiresat the second time); and calculate a difference between thislongitudinal distance and a predefined target longitudinal offset foropening the blades 132 prior to reaching a target plant. For example,the predefined target longitudinal offset can be selected based on: asize or age of the crop being weeded (e.g., a larger predefined targetlongitudinal offset for larger or older plants); a type of topsoil(e.g., larger predefined target longitudinal offset for rockier soil);or dampness of the topsoil (e.g., larger predefined target longitudinaloffset for wetter topsoil); etc. The autonomous machine 100 can thencalculate a revised open time correction for a next target plant to passthrough the first weeding module 130 in Block S270 by: multiplying thisdifference by a speed of the autonomous machine 100 at the second time;and subtracting from this product the open time correction implementedfor the current (i.e., the first) target plant.

Similarly, the autonomous machine 100 can calculate a longitudinaldistance from tips of the blades 132 to the center of the stalk of thefirst target plant as or just before the autonomous machine 100 triggersthe blade actuator to close the blades 132 (e.g., at or just before theclose timer expires at a third time). The autonomous machine 100 canthen: calculate a difference between this longitudinal distance and apredefined target longitudinal offset for closing the blades 132 afterpassing a target plant; and calculate a revised close time correctionfor the next target plant to pass through the first weeding module 130in Block S290 by dividing this difference by a speed of the autonomousmachine 100 at the third time and subtracting from this product theclose time correction implemented for the current (i.e., the first)target plant.

6.5.5 Next Plant in the First Crop Row

As the autonomous machine 100 navigates along crop rows in theagricultural field and as target plants enter the light module 110 andare detected in entry images, the autonomous machine 100 can repeat theforegoing processes to revise lateral offset correction, open timecorrection, and close time correction values implemented by theautonomous machine 100 for each subsequent target plant based oninteraction between the weeding module 130 and preceding target plants.In particular, the autonomous machine 100 can implement closed-looptechniques to feed time and lateral positioning measurements—recordedduring actuation of the weeding module 130 to remove weed around a firsttarget plant—forward to adjust an open time, a close timer, and alateral position for the weeding module 130 for a next target plant toenter the tool housing 120 behind the first target plant; and so onuntil the autonomous machine 100 completes the weeding operation overthe row of crops or the entire agricultural field.

6.6 Other Crop Rows

Furthermore, the autonomous machine 100 can execute multiple instancesof the foregoing Blocks of the methods S100 or S200 in parallel: todetect target plants approaching other weeding modules 130—installed inthe tool housing 120—substantially concurrently; to drive the weedingmodules 130 to their lateral offset positions substantiallysimultaneously; and to trigger blade actuators in the weeding modules130 to open their corresponding blades 132 independently upon expirationof their corresponding open timers; and to calculate new corrections—inthe corresponding crop rows—entering the tool housing 120. In oneimplementation, the autonomous machine 100 can include a single rearcamera 136 capable of recording interaction images for multiple toolmodules in the tool housing 120 in order to reduce the number of rearcameras 136 included in the autonomous machine 100.

6.7 Other Operations

The autonomous machine 100 can implement similar methods and techniquesto set timers for actuating tool modules of other types, to set lateralpositions of these tool modules for passing target plants, and to thuscontrol actuation of these tool modules in order to selectively addresstarget plants and non-target plants (e.g., weeds) in an agriculturalfield. For example, the autonomous machine 100 can implement theforegoing methods and techniques to: laterally align a fertilizing headin a fertilizer module to a next target plant in a crop row; dispensefertilizer from the fertilizer module directly onto or in the immediatevicinity of the next target plant; record exit images to detect offsets;and apply corrections to improve subsequent fertilizing operations.

The systems and methods described herein can be embodied and/orimplemented at least in part as a machine configured to receive acomputer-readable medium storing computer-readable instructions. Theinstructions can be executed by computer-executable componentsintegrated with the application, applet, host, server, network, website,communication service, communication interface,hardware/firmware/software elements of a user computer or mobile device,wristband, smartphone, or any suitable combination thereof. Othersystems and methods of the embodiment can be embodied and/or implementedat least in part as a machine configured to receive a computer-readablemedium storing computer-readable instructions. The instructions can beexecuted by computer-executable components integrated bycomputer-executable components integrated with apparatuses and networksof the type described above. The computer-readable medium can be storedon any suitable computer readable media such as RAMs, ROMs, flashmemory, EEPROMs, optical devices (CD or DVD), hard drives, floppydrives, or any suitable device. The computer-executable component can bea processor but any suitable dedicated hardware device can(alternatively or additionally) execute the instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the embodiments of the invention without departing fromthe scope of this invention as defined in the following claims.

We claim:
 1. A method for autonomously weeding crops in an agriculturalfield comprises, at an autonomous machine autonomously navigating alongcrop rows in the agricultural field: recording a first image of a groundarea below a light module arranged proximal a front of the autonomousmachine; detecting a location of a first target plant based on the firstimage; calculating a first opening location for the first target plantlongitudinally offset from the location of the first target plant andlaterally aligned with the location of the first target plant; driving afirst weeding module to laterally align a lateral reference position ofthe first weeding module with the first opening location, the firstweeding module arranged in a tool housing behind the light module;tracking the first opening location relative to a longitudinal referenceposition of the first weeding module; in response to the longitudinalreference position of the first weeding module longitudinally aligningwith the first opening location, actuating blades of the first weedingmodule to an open position of the blades of the first weeding module;recording a second image of a ground area below the tool housing; inresponse to detecting the blades of the first weeding module in the openposition in the second image: calculating a first longitudinal offsetbetween the first opening location and the longitudinal referenceposition of the first weeding module, based on the second image; andcalculating a first lateral offset between the first opening locationand the lateral reference position of the first weeding module, based onthe second image.
 2. The method of claim 1: further comprising recordinga sequence of images of the ground area below the tool housing, thesequence of images comprising the second image; further comprising, inresponse to detecting the blades of the first weeding module in the openposition in the second image, selecting the second image from thesequence of images; and wherein detecting the blades of the firstweeding module in the open position in the second image furthercomprises: detecting the blades of the first weeding module in the openposition by detecting a position of blade fiducials of the first weedingmodule in the second image, the blade fiducials of the first weedingmodule mechanically constrained relative to the blades of the firstweeding module; and in response to the position of the blade fiducialsof the first weeding module in the second image indicating the openposition, detecting blades of the first weeding module in the openposition.
 3. The method of claim 1, wherein recording the second imageof the ground area below the tool housing further comprises recordingthe second image of the ground area below the tool housing via a firstrear camera mounted to the first weeding module.
 4. The method of claim3, further comprising: calibrating a pixel location of the longitudinalreference position of the first weeding module in a field of view of thefirst rear camera, the pixel location of the longitudinal referenceposition of the first weeding module coinciding with an intersectionpoint of the blades of the first weeding module; and calibrating a pixellocation of the lateral reference position of the first weeding modulein the field of view of the first rear camera, the pixel location of thelateral reference position of the first weeding module coinciding withthe intersection point of the blades of the first weeding module.
 5. Themethod of claim 4: further comprising, detecting a pixel location of thefirst target plant in the second image; further comprising, calculatinga pixel location of the first opening location based on the pixellocation of the first target plant in the second image; and whereincalculating the first longitudinal offset between the first openinglocation and the longitudinal reference position of the first weedingmodule further comprises calculating the first longitudinal offsetbetween the first opening location and the longitudinal referenceposition of the first weeding module based on the pixel location of thefirst opening location and the pixel location of the longitudinalreference position in the second image; wherein calculating the firstlateral offset between the first opening location and the lateralreference position of the first weeding module further comprisescalculating the first lateral offset between the first opening locationand the lateral reference position of the first weeding module based onthe pixel location of the first opening location and the pixel locationof the lateral reference position in the second image.
 6. The method ofclaim 3, further comprising: calibrating an open position of bladefiducials of the first weeding module, the open position of the bladefiducials of the first weeding module indicating the open position ofthe blades of the first weeding module; and calibrating a closedposition of blade fiducials of the first weeding module, the closedposition of the blade fiducials indicating a closed position of theblades of the first weeding module.
 7. The method of claim 1, furthercomprising: recording a third image of the ground area below the lightmodule; detecting a location of a second target plant based on the thirdimage; calculating a second opening location for the second target plantlongitudinally offset from the location of the second target plant andlaterally aligned with the location of the second target plant;calculating a corrected second opening location by offsetting the secondopening location by the first longitudinal offset and the first lateraloffset; driving the first weeding module to laterally align the lateralreference position of the first weeding module with the corrected secondopening location; tracking the corrected second opening locationrelative to the longitudinal reference position of the first weedingmodule; in response to the longitudinal reference position of the firstweeding module longitudinally aligning with the corrected second openinglocation, actuating the blades of the first weeding module to the openposition; recording a fourth image of the ground area below the toolhousing; in response to detecting the blades of the first weeding modulein the open position in the fourth image; calculating a secondlongitudinal offset between the second opening location and thelongitudinal reference position of the first weeding module, based onthe fourth image; and calculating a second lateral offset between thesecond opening location and the lateral reference position of the firstweeding module, based on the fourth image.
 8. The method of claim 7,further comprising: averaging the first longitudinal offset and thesecond longitudinal offset to calculate a longitudinal correction;averaging the first lateral offset and the second lateral offset tocalculate a lateral correction; recording a fifth image of the groundarea below the light module; detecting a location of a third targetplant based on the fifth image; calculating a third opening location forthe third target plant longitudinally offset from the location of thethird target plant and laterally aligned with the location of the thirdtarget plant; calculating a corrected third opening location byoffsetting the third opening location by the longitudinal correction andthe lateral correction; driving the first weeding module to laterallyalign the lateral reference position of the first weeding module withthe corrected third opening location; tracking the corrected thirdopening location relative to the longitudinal reference position of thefirst weeding module; in response to the longitudinal reference positionof the first weeding module longitudinally aligning with the correctedthird opening location, actuating the blades of the first weeding moduleto the open position; recording a sixth image of the ground area belowthe tool housing; in response to detecting the blades of the firstweeding module in the open position in the sixth image; calculating athird longitudinal offset between the third opening location and thelongitudinal reference position of the first weeding module, based onthe sixth image; and calculating a third lateral offset between thethird opening location and the lateral reference position of the firstweeding module, based on the sixth image.
 9. The method of claim 1,further comprising: detecting a location of a second target plant basedon the first image; calculating a second opening location for the secondtarget plant longitudinally offset from the location of the secondtarget plant and laterally aligned with the location of the secondtarget plant; driving a second weeding module to laterally align alateral reference position of the second weeding module with the secondopening location, the second weeding module arranged in the tool housingbehind the light module adjacent to the first weeding module; trackingthe second opening location relative to a longitudinal referenceposition of the second weeding module; in response to the longitudinalreference position of the second weeding module longitudinally aligningwith the second opening location, actuating blades of the second weedingmodule to an open position of the blades of the second weeding module;recording a third image of a ground area below the tool housing; inresponse to detecting, in the third image, the blades of the secondweeding module in the open position of the blades in the second weedingmodule in the third image: calculating a second longitudinal offsetbetween the second opening location and the longitudinal referenceposition of the second weeding module, based on the third image; andcalculating a second lateral offset between the second opening locationand the lateral reference position of the second weeding module, basedon the third image.
 10. The method of claim 1: further comprisingcalculating an opening distance based on plant features of the firsttarget plant; and wherein calculating the first opening location for thefirst target plant longitudinally offset from the location of the firsttarget plant and laterally aligned with the location of the first targetplant further comprises calculating the first opening location for thefirst target plant offset by the opening distance from the location ofthe first target plant and laterally aligned with the location of thefirst target plant.
 11. The method of claim 10, wherein calculating theopening distance based on plant features of the first target plantfurther comprising calculating the opening distance based on a speciesof the first target plant and an age of the first target plant.
 12. Themethod of claim 1, further comprising: calculating a first closinglocation for the first target plant longitudinally offset from andbehind the location of the first target plant and laterally aligned withthe location of the first target plant; in response to the longitudinalreference position of the first weeding module longitudinally aligningwith the first closing location, actuating the blades of the firstweeding module to a closed position; recording a third image of a groundarea below the tool housing; in response to detecting the blades of thefirst weeding module in the closed position in the third image:calculating a second longitudinal offset between the first closinglocation and the longitudinal reference position of the first weedingmodule, based on the third image; and calculating a second lateraloffset between the first closing location and the lateral referenceposition of the first weeding module, based on the third image.
 13. Themethod of claim 12: further comprising calculating an opening distancebased on plant features of the first target plant; further comprisingcalculating a closing distance based on plant features of the firsttarget plant; wherein calculating the first opening location for thefirst target plant longitudinally offset from the location of the firsttarget plant and laterally aligned with the location of the first targetplant further comprises calculating the first opening location for thefirst target plant offset by the opening distance from the location ofthe first target plant and laterally aligned with the location of thefirst target plant; and wherein calculating the first closing locationfor the first target plant longitudinally offset from and behind thelocation of the first target plant and laterally aligned with thelocation of the first target plant further comprising calculating thefirst closing location for the first target plant offset by the closingdistance from the location of the first target plant and laterallyaligned with the location of the first target plant.
 14. The method ofclaim 1: further comprising recording a sequence of images of the groundarea below the light module, the sequence of images comprising the firstimage; wherein detecting the location of the first target plant based onthe first image further comprises, for each image in the sequence ofimages detecting an estimated location of the first target plant basedon the image; further comprising, in response to detecting the estimatedlocation of the first target plant at a longitudinal center of the firstimage, selecting the first image from the sequence of images; andwherein calculating the first opening location for the first targetplant longitudinally offset from the location of the first target plantand laterally aligned with the location of the first target plantfurther comprises, in response to selecting the first image from thesequences of images, calculating the first opening location for thefirst target plant longitudinally offset from the estimated location ofthe first target plant and laterally aligned with the estimated locationof the first target plant.
 15. The method of claim 1, further comprisinggenerating a first set of visual features of the first image; whereincalculating the first longitudinal offset between the first openinglocation and the longitudinal reference position of the first weedingmodule, based on the second image; and calculating the first lateraloffset between the first opening location and the lateral referenceposition of the first weeding module, based on the second image furthercomprises; generating a second set of visual features of the secondimage; mapping the first set of visual features to the second set ofvisual features to generate a set of like features in the second image;detecting an estimated location of the first target plant in the secondimage based on the set of like features between the first set of visualfeatures and the second set of visual features; and calculating anestimated first opening location based on the estimated location of thefirst target plant; calculating the first longitudinal offset betweenthe first opening location and the longitudinal reference position ofthe first weeding module, based on the estimated first opening location;and calculating the first lateral offset between the first openinglocation and the lateral reference position of the first weeding module,based on the estimated first opening location.
 16. The method of claim1, wherein tracking the first opening location relative to alongitudinal reference position of the first weeding module furthercomprises: tracking a motion of the autonomous machine relative to theagricultural field; and transforming the first opening location based onthe motion of the autonomous machine relative to the agricultural field.17. The method of claim 1, wherein recording the first image of theground area below the light module arranged proximal the front of theautonomous machine further comprises recording the first image of theground area below the light module arranged proximal the front of theautonomous machine via a front camera mounted to a chassis of theautonomous machine within the light module.
 18. A method for performingan agricultural operation in an agricultural field comprises, at anautonomous machine autonomously navigating along crop rows in theagricultural field: recording a first image of a ground area below alight module arranged proximal a front of the autonomous machine;detecting a location of a first target plant based on the first image;driving a first operating module to laterally align a lateral referenceposition of the first operating module with the location of the firsttarget plant, the first operating module arranged in a tool housingbehind the light module; tracking the location of the first target plantrelative to a longitudinal reference position of the first operatingmodule; in response to the longitudinal reference position of the firstoperating module longitudinally aligning with the location of the firsttarget plant, actuating the first operating module to perform theagricultural operation; recording a second image of a ground area belowthe tool housing; in response to detecting a fiducial of the firstoperating module in the second image, the fiducial indicating aperformance of the agricultural operation: calculating a firstlongitudinal offset between the location of the first target plant andthe longitudinal reference position of the first operating module, basedon the second image; and calculating a first lateral offset between thelocation of the first target plant and the lateral reference position ofthe first operating module, based on the second image.
 19. The method ofclaim 18: wherein actuating the first operating module to perform theagricultural operation further comprises actuating the first operatingmodule to perform a weeding operation; and wherein calculating the firstlongitudinal offset between the location of the first target plant andthe longitudinal reference position of the first operating module; andcalculating the first lateral offset between the location of the firsttarget plant and the lateral reference position of the first operatingmodule further comprises, in response to detecting a fiducial of thefirst operating module in the second image, the fiducial indicating aperformance of the agricultural operation and mechanically constrainedrelative to an end effector of the first operating module.
 20. A methodfor autonomously weeding crops in an agricultural field comprises, at anautonomous machine autonomously navigating along crop rows in theagricultural field: at a first time, recording a first image of a groundarea in a light module arranged proximal a front of the autonomousmachine; detecting a location of a target plant in the first image;calculating a longitudinal distance from the location of the targetplant to tips of closed blades in a weeding module arranged in a toolhousing behind the light module; estimating a duration of time at whichthe target plant will reach a longitudinal position of the closed bladesof the weeding module based on the longitudinal distance and a speed ofthe autonomous machine at the first time; initiating a timer for a sumof the duration of time and an open time correction; calculating alateral offset from the location of the target plant at the first timeto a lateral reference position of the weeding module; driving theweeding module to a lateral position offset from the lateral referenceposition by a sum of the lateral offset and lateral offset correction;in response to expiration of the timer at a second time, triggering theblades in the first weeding module to open for an open duration; and inresponse to conclusion of the open duration at a third time, triggeringthe blades in the weeding module to close.